Computer application launching

ABSTRACT

A computer-implemented user interface method includes displaying on a mobile device desktop a background image and one or more foreground items over the background image, receiving a user input to pan across the desktop, and panning the background images and one or more foreground images in a parallax manner relative to each other.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional application of U.S. patent applicationSer. No. 14/858,848 filed on Sep. 18, 2015, which is a continuation ofU.S. patent application Ser. No. 13/463,383 filed on May 3, 2012 (nowissued as U.S. Pat. No. 9,141,272), which is a divisional application ofU.S. patent application Ser. No. 12/474,238 filed on May 28, 2009 (nowissued as U.S. Pat. No. 8,266,550), which claims the benefit of U.S.Provisional Application No. 61/056,825, filed on May 28, 2008, theentire contents of each of which are hereby incorporated by reference.

TECHNICAL FIELD

This document relates to systems and techniques for organizing elementson a computer desktop and for executing actions associated with suchelements.

BACKGROUND

People spend hours at a time with their electronic devices—computers,telephones, music players, and the like. They like best those devicesthat are intuitive to use and whose interactions best meet theirexpectations regarding how machines should work. They interact withelectronics through inputs and outputs from the devices, where theoutputs generally are provided audibly and/or on a flat graphicaldisplay screen, and the inputs may occur via touch screens, joysticks,mice, 4-directional keypads (e.g., d pads), and other such inputmechanisms.

Fine distinctions in user interfaces can make a world of difference in auser's experience with a device. Intuitive user interface interactionsmay be at a premium for mobile devices, where display sizes aregenerally limited, as are input mechanisms (e.g., full size keyboardsmay be hard to implement because of space constraints, or may be hard touse when a user cannot easily use both hands with their computingdevice).

SUMMARY

This document describes systems and techniques that may be employed bysomeone who wants to interact with applications on a computing devicelike a mobile telephone that has a touch screen user interface, andperhaps other input mechanisms such as trackballs. In general, thetechniques relate to a desktop interface for an operating system of sucha device. The desktop may generally be a “home” location for anoperating system, and can include a number of selectable icons that,when selected, cause certain defined actions to be performed. Forexample, when an icon for a software application is selected, theapplication may be launched, and when an icon for a certain type ofdocument is selected, a software application associated with thedocument may be launched with the document opened in the application.

In certain examples shown here, the types of actions that may beassociated with desktop icons may be increased dramatically byassociating each icon with a URI in the system that points to aparticular operating system object. Such objects can include executableapplications, database records such as a contact of the user, files,persistent searches (e.g., searches having multiple terms and connectorsthat are saved over time so that a user need not enter the search overand over), and other such targets.

Consistent with this idea of treating various display elements on adesktop in similar manners, each of the various types of graphicalelements may be assigned to a single layer on the desktop. For example,icons for shortcuts, and gadgets or widgets, may all be on the samelayer. Objects that share a common layer are treated as being part of acommon group by the operating system rather than as being a subset ofone particular layer. In addition, for this reason, objects on a singlelayer are treated similarly for display purposes, and they may thus notbe allowed to overlap visually with each other on a display (at leastnot fully overlap).

The desktop may span across multiple screens of the mobile device so asto provide additional space for placing desktop elements where thedisplay is small. A user may move back and forth between the differentdisplays, and may drag elements from one display to another. The desktopmay also include a background that holds a wallpaper image, in afamiliar and known manner. The background may stretch across all of themultiple displays so as to give the user an visual baseline so that theymore readily understand which display they are currently viewing—i.e.,they understand their positional context on the multi-screen desktop.When a user pans from one display to another, the wallpaper orbackground may move at a rate that is slower that elements on thedesktop in the foreground, such as icons, move, so as to provide athree-dimensional parallax effect to the panning effort.

In addition, the manner in which user feedback is provided on the devicescreen in response to user inputs may vary depending on the type ofmechanism that the user has employed to provide the input. Such changesmay occur automatically and in the middle of the user's use of thedifferent mechanisms. For example, when a user employs a pointer input,which is a device such as a joystick or trackball that is separate fromthe display but permits control of an object such as an arrow or otherpointer on the screen, selections of elements on the screen may beindicated by highlighting the objects and snapping from one object t thenext. If a user then switches to providing input to the touch screen(e.g., by pressing a finger to the screen and moving it), the previoushighlighting can be removed and replaced by feedback that relates to thetouch input, such as by highlighting other elements on the desktop thatwere touched and dragged across, or by drawing a window around suchobjects. Such a transition may occur automatically, by the systemrecognizing a change in active input mechanism.

In certain implementations, such systems and technique may provide oneor more advantages. For example, with multiple screens that interact inthese manners, a user may be provided with a relatively large number oficons or similar objects on a desktop without having to shrink eachobject until it can no longer be seen on a small screen. Also, a usermay easily customize their desktop so that objects that are mostimportant to them can be reached quickly and intuitively. The reactionsof a system to a user may also be natural, including reaction when theuser moves from one input mechanism to another. In addition, gadgets orwidgets may be created and accessed more readily, and may be integratedwith a user's desktop to provide a highly customizable experience. Eachof these features may be provided in a visually pleasing manner, and ina way that can incorporate both touch screen and pointer (e.g.,trackball) input.

In one implementation, a computer-implemented user interface method isdisclosed. The method comprises displaying on a mobile device desktop abackground image and one or more foreground items over the backgroundimage, receiving a user input to pan across the desktop, and panning thebackground images and one or more foreground images in a parallax mannerrelative to each other. The foreground items may be panned at a rateapproximately 150% of the rate at which the background is panned. Themethod can also include panning to automatic snap points on amulti-screen desktop, and displaying foreground items from a screen thatis adjacent to a screen on which the user input to pan is received.Moreover, the user input to pan can be received in the form of a fingerflick on a touch screen input mechanism.

In another implementation, a computer-implemented user interface methodcomprises displaying on a computer desktop one or more program modules,and displaying on the computer desktop one or more icons that, whenselected by a user, cause the execution of objects stored on thecomputer. The one or more program modules are displayed on a same layerof the desktop as are the one or more icons. In addition, the one ormore icons may each include references to URIs, and selection of an iconmay cause the device to be pointed to a URI associated with the icon.

The details of one or more embodiments are set forth in the accompanyingdrawings and the description below. Other features and advantages willbe apparent from the description and drawings, and from the claims.

DESCRIPTION OF DRAWINGS

FIG. 1A shows portrait and landscape displays of a mobile devicedesktop.

FIG. 1B shows a progression of displays for a mobile device desktop.

FIGS. 2A and 2B show manipulation of graphical elements on a mobiledevice display.

FIGS. 2C and 2D show schematically techniques for panning across adesktop space.

FIG. 2E shows portrait and landscape displays for arranging desktopicons.

FIG. 3 is a schematic diagram of a system that provides user interactionin response to touch screen inputs.

FIGS. 4A-4D are flow charts of example processes for interacting withgraphical elements on a computer desktop.

FIG. 5 is a schematic representation of an exemplary mobile device thatimplements embodiments of the notification techniques described herein.

FIG. 6 is a block diagram illustrating the internal architecture of thedevice of FIG. 5.

FIG. 7 is a block diagram illustrating exemplary components of theoperating system used by the device of FIG. 5.

FIG. 8 is a block diagram illustrating exemplary processes implementedby the operating system kernel of FIG. 7.

FIG. 9 shows an example of a computer device and a mobile computerdevice that can be used to implement the techniques described here.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

This document describes systems and techniques by which mobile devicesmay interact with users of such devices. In particular, interaction withelements on a computer desktop are described here, such as the creationof, arrangement of, display of, and interaction with, icons on acomputer desktop. In addition, navigation around a computer desktop isalso described, as are other related features for a computing devicesuch as a mobile smart phone.

FIG. 1 shows portrait and landscape displays of a mobile device desktop.In the figure, a mobile device 102 is shown as displaying a number oficons, such as icon 110 on a computer desktop in a typical and familiarmanner. The icons here have been genericized, and the genericinformation would, in normal circumstances, be replaced by an actualname of an application associated with an icon, and a computer graphicto represent the icon readily to a user of the device 102.

Although shown as icons for computer applications, the icons, such asicon 110, may be associated with a wide variety of computing objects, asdescribed more fully below. For example, an icon could point to a folderof files, to a file, to a music source, to a contact in a contact listof a user, to a persistent query that, when it is activated, sends apredetermined query string to a remote search engine, or to any otherappropriate operating system object.

An applications area 104 is defined on the computer desktop. Theapplications area 104 includes an icon for every computer applicationthat is currently installed on the device 102 (or at least a sub-set ofthe installed applications). The applications area 104 thus serves as aplace to which a user may turn when attempting to launch an applicationor when determining whether an application is available on a device 102.Nine icons in the application area 104, such as icon 106, are displayedin a manner similar to that for icons outside the application area 104,and may be interacted with in a similar manner. For example, as iswell-known, a user's selection of an icon associated with a computerapplication, such as by the user clicking or tapping on the icon, maycause the computer application associated with the icon to be launchedand made active on the device 102.

Certain icons, such as icon 114, are displayed in a bottom bar 112. Theicons in the bottom bar 112 may be interacted with in a manner similarto other icons (e.g., clicking on them may cause their associatedapplications or other objects to launch), but they are generally madealways available in the bottom bar 112, regardless of where the user isotherwise positioned on the desktop. For example, as will be shownfurther below, the desktop may have multiple screens or displays thatare not currently visible to a user, and the user may pan back and forthbetween those multiple screens. Icons such as icon 110 may thus becomehidden from immediate view when the user pans to a different display.However, icons, such as icon 114, in the bottom bar 112 will remainanchored at the bottom of the display even when the user pans.

The two devices shown in FIG. 1A represent a single device that istransformed between portrait presentation mode and landscape mode at apoint in time. Such a transformation may occur automatically, forexample, when a users slides open a slider telephone so as to indicatean intent to use it in landscape mode, using the device keyboard. Onesuch change in displays from portrait to landscape mode includes atransformation of the bottom bar 112 into a sidebar. Such atransformation occurs so as to maintain the bar 112 along a short sideof the display, and to thus avoid altering the overall layout of thedisplay. For example, as shown, there is space for the bar 112 to showfour icons along the short side of the display. By moving the positionof the bar 112, the same four icons may generally be displayed in theother mode also.

In addition, the display of icons such as icon 114 relative to bar 112has changed slightly between the two modes. For example, in portraitmode, the icons extend upward onto the desktop slightly from bar 112 soas to provide a form of three-dimensional effect of the icons floatingover the bar 112. In landscape mode, the icons are shown wholly insidethe bar 112, or could alternatively be shown having their left edgeshanging over the edge of the bar 112.

The display of the applications area 104 also changes slightly when thedevice is transformed from portrait mode to landscape mode. Inparticular, the title bar 106 for the applications area 104 may be movedfrom the top of the area 104 to the side of the area 104. Again, suchmovement helps ensure that the area 104 does not become overlycompressed along the shortest dimension of the display, by moving thetitle bar 106 into the longer dimension of the display at all times,where there is ample room for it.

Two points about the general layout of the icons on the desktop whentransforming from portrait mode to landscape mode are notable here.First, the icons are redistributed evenly across the space so as to fillup the appropriate dimensions. As a result, for example, there may beadded room for icon names in landscape mode, so that the applicationname for each icon may need to take two lines in portrait mode, but mayexpand to fill only a single line in landscape mode, as additionalhorizontal space is allocated to each icon. Second, the icons are notsimply rotated on their own individual axes during the transformationfrom one mode to another in this example. If they were, the applicationsarea 104 would be located in the lower left corner of the display inlandscape mode after being converted from the portrait mode. Instead,the relative positioning of the icons on the desktop, apart from theicons in bar 112, is maintained between the modes. Thus, for example,the applications area 104 remains in the upper left corner of thedisplay in both modes, with the main desktop items wrapped around it ina similar manner, and the spacing is simply readjusted.

The particular arrangement of icons between portrait mode and landscapemode can be seen even more clearly at FIG. 2E. In this figure, variousicons are lettered so as to better demonstrate the relative positioningof icons in portrait mode versus landscape mode. As shown, a device 260is initially in a portrait mode arranged similarly to the layout shownin FIG. 1A. In the transformation from portrait mode to landscape mode,an applications area 262 is widened and shortened, and the icons in theapplication area 262 are redistributed across the changed area 262, butthe orientation and arrangement of the icons relative to each other doesnot change. Likewise, an icon on the main desktop, such as icon 264,remains in its initial location, in the lower left corner of the maindesktop. An icon 266 in a bottom bar changes its position so that thebottom bar becomes a sidebar, but the icon 266 maintains its positionrelative to other icons in the bar, moving from the leftmost position inportrait mode to the bottommost position in landscape mode.

Also, in this example, the icons are snapped to discrete positions onthe desktop, where there is no, or minimal, overlap between adjacenticons. Thus, as shown below, a user may move an icon freely around thedesktop and may be given the impression that the icon may occupy aninfinite number of locations on the desktop, but the icon may besmoothly snapped to a centerpoint of one of the locations when the userreleases it. Alternatively, the icons may be permitted to be placedwithout regard to snapping to a particular grid, though such anarrangement may result in a messy appearance for the desktop.

FIG. 1B shows a progression of displays for a mobile device desktop. Theexample shown in the figures demonstrates interaction with icons in anapplications area 122. The application area 122 may be less than theentire displayed desktop in size, and its size and/or shaped may bechanged by a user, to show more or fewer of the items that are in theapplication area 122. In particular, the application area 122 may haveits dimensions changed by a user, as discussed below, such as by theuser dragging on a corner of the application area 122 to expand orcontract it. As a result, it may not be possible for the applicationarea 122 to display icons for all available applications at the sametime. In such a situation, certain icons may be hidden off the edge ofthe application area 122, and a user may pan within the application area122 itself. In one example, the size of the application area 122 may bebounded between a 1×2 area or 2×1 area at the smallest and a 4×4 area atthe largest.

In the leftmost display, the application area 122 has a 4×3 size, whilein the middle display, it has a 4×4 size and thus occupies all the spaceof the currently displayed bare desktop on the screen. When the area 122is expanded, additional icons that were previously hidden outside theedge of the area 122 (i.e., off the bottom) may be displayed, much likeadditional icons are traditionally shown when a window is enlarged. Theleftmost display may represent a default middle screen for a desktopthat is displayed when a device is first activated, as it shows to auser all of the three different forms of icons available in thisexample: icons on the bare desktop itself, icons in a sub-area of thedesktop (e.g., in a box), and icons in the bottom bar. When theapplications area 122 is expanded, the icons on the desktop itself maybe covered and hidden by the application area 122 or may be pushed ontoadjacent displays that a user may reach by swiping sideways across thedesktop.

The rightmost display in FIG. 1B shows the result of a user scrollingwithin the application area 122. In particular, a user can touch orflick-scroll down the application area 122 by touching at a low portionof the area 122 and flicking their finger upward while releasing it fromthe screen. Such a move will seem to transfer momentum to the icons andresult in them scrolling upward at a rate generally proportional to thespeed with which the user flicked. As the icons moved upward, the userscrolls downward through the icons.

The particular positioning of the user within the area 122 of icons thatcannot all be displayed at one time, can be seen in this example by twointerface elements. First, in the middle screen, the lowest icons in thearea 122 are faded to black at their bottom edges, whereas in therightmost screen, the highest icons are faded to black along their upperedges. Such a feature indicates to a user that there are additionalitems below or above the display, respectively, by simulating a wheelcovered with icons that disappears into the darkness. Second, a scrollbutton 124 or scroll handle is displayed at the right edge of therightmost display. The scroll button 124 may appear whenever thecontents of an area are larger than the area itself, or may be displayedmomentarily when the user flicks within an area to scroll. The user maythen “grab” the button 124 to scroll up and down in a familiar manner.

FIGS. 2A and 2B show manipulation of graphical elements on a mobiledevice display. In general, the figures show the manner in which iconsand program modules such as gadgets or widgets may be moved around adesktop surface. The example here focuses on the interaction of anapplication area 202, a program module 206, and several icons 203, 204,208, across three different versions of the same mobile device display.

In the leftmost display, a mobile device 200 shows a 2×2 applicationarea 202 that contains four icons. The four icons may represent all ofthe applications that are currently loaded and available for executionon the device 200. A user selection of any of the icons, such as bytapping on a touch screen over the appropriate icon or moving a pointerto the area with a pointing device, such as a trackball, and clickingwith the pointing device, may cause the underlying application tolaunch. In this example, however, a user has long-pressed on the lowerright icon to select it (but has not released their finger yet), andthen dragged the icon 204 out of the application area 202 and onto themain desktop.

Such selection by the user may be accompanied by various feedbackmechanisms to the user. For example, when the user has long pressed fora sufficient time in order to register a selection of the icon 204, theicon may increase slightly in size (e.g., by 10-30 percent) to make itlook like the item is closer and has thus been picked up by the user,may change colors such as a by having its colors invert, or may beincreased slightly in brightness, to indicate to the user visually thatthe icon is ready to be moved or otherwise manipulated. The size orother highlighting of the object may then be removed after the user hasmoved and release the object, so that the object returns to being anormal size as part of the desktop.

Such feedback may shift if a user changes the input mechanism that theyare using in order to interact with the display. For instance, a usermay initially navigate around the display using a pointer device such asa trackball. Because such interaction does not have a direct connectionbetween a position of the device (i.e., a trackball can be rolledinfinitely in any direction) and a location on the display, a pointercan be displayed on the screen to reflect the current position of theuser on the screen. Such motion by a user may also occur without apointer being displayed, but instead with objects on the display beinghighlighted individually as the user manipulates the pointing device.For example, when a user initially opens a desktop covered in icons, oneof the icons can be highlighted, such as by displaying it in a brightermanner than other icons, displaying it in an inverse coloring, ordrawing a bright line around the periphery of the icon (e.g., showing alight-colored square around the icon). Such an indication can show thatthe particular icon is currently the active icon, and that the icon canbe selected, or that actions specified by the user will be performed onthat icon or the application or other object that is represented by theicon. As the user moves the pointing device, the highlighting can jumpor snap from one object to the next object in the direction of theuser's movement, so that when the presumed location of the user on thedisplay as they move a pointing device passes a midpoint between the twoicons, the highlighting disappears from the first object and appears onthe second object.

Also, a user may then click on a selection button that corresponds tothe pointing device so as to select an object on the screen. The usercan subsequently move the pointing device to move the pointer acrossmultiple objects, such as icons on a desktop so as to move theoriginally selected object to a new location or to select across themultiple objects. In the latter circumstance, the selected objects caneach in turn be highlighted such as in the manners discussed above, toindicate that they are now the focus of the device.

The interface may change if the user takes their finger off the pointingdevice and moves it to the surface of the touch screen (which is not apointing device as that term is used here). In such a situation, therewill not be a need for an pointer on the display to indicate where theuser is currently located, because the user's fingertip will necessarilybe at that location. As a result, when the user first contacts thescreen, the pointer, if it was previously displayed, may be removed fromthe display. Also, any selected icons can, in appropriate circumstances,become un-highlighted, and removed from a list of items that will beaffected by a current user action on the device. Such un-highlighting ofthe objects may also wait until the user contacts the surface of thescreen and also moves a predetermined amount on the screen (to indicate,e.g., that the user may want to perform a different selection action onthe device). In addition, the un-highlighting may not occur if the userselects a particular area of the screen. For example, a certain area ofthe screen may contain buttons whose selection causes an action to beperformed, while another area may include selectable icons or otherobjects. A user could thus use a pointing device to select objects inthe latter area, and follow up by selecting on the touch screen one ofthe buttons in the former area—without having the objects becomeunselected.

In addition, tactile feedback may be provided to the user, such as bythe device 200 shaking slightly when the icon 204 has been selected, andalso shaking as the user passes each location in a grid on the desktop.Alternatively, similar tactile feedback may be provided to indicateother events to a user, including so that a user need not have a displayor look at the display. Also, audio feedback may be provided for similarpurposes, and voice input may be received and translated by the system.The tactile feedback can also occur only when the interface is snappingbetween objects, such as when the user is employing a pointing devicefor input, so as to simulate the snapping motion, but may not move whenthe screen is being used for input.

Referring again to particular objects on the display, a program module206 takes the form of a familiar clock/date module that sits on thesurface of the desktop. The module 206 may be located on the same layeras the area 202 and the icons 203, 204. As a result, the module 206 maybe treated at the same hierarchical level by the operating system, asare the other objects, and may displace such objects if it is moved intothe area of the objects. In certain instances, although the module 206appears to be free-floating over the desktop, it may in actuality besnapped into a grid on the desktop. Thus, for example, in this example,the module 206 occupies the space of four grid positions on the desktop.The module 206 may also be dragged by a user to be larger or smaller,and may, as a result, take up more or fewer grid positions. When themodule 206 is dragged be larger, adjacent icons may be pushed out of theway and across the desktop.

The middle display shows two relevant changes from the leftmost display.First, the user has changed the form factor of area 202, from a 2×2arrangement to a 1×3 arrangement. Second, the display shows a selectionof icon 204 to combine icon 204 with icon 203 to create a new box orarea 210. In particular, when a user long presses on one icon and dragsit on top of another icon, the two icons may be grouped together, and anew box or area may be automatically formed around them. In a familiarmanner, the new box or area may be given a generic name, and a cursormay be initially placed over that name so that the user may convenientlytype in a custom name for that box or area 210. In contrast, where a boxor area contains only two icons, and a user drags one of the icons outonto the bare desktop, the box or area may, in certain examples,automatically disintegrate, so that both of the last icons move to thedesktop, with the last remaining icon from the box occupying a positionon the desktop that is in alignment with its former position in the boxor area.

The rightmost display in FIG. 2A shows an example of the movement of anicon into the bottom bar. As noted above, the bottom bar serves as alocation to anchor certain icons so that they may be readily accessedfrom any of the multiple screens across a desktop. In generalsituations, for example, a user may desire to place frequently accessedsearches, documents, or applications in the bar.

The placement of an icon 212 is shown by arrows in the rightmostdisplay. In particular, a user is shown as having long pressed on icon212 in area 202, and then to have dragged the icon 212 down into thebottom bar and over the top of a pre-existing icon 214. In this manner,the user has indicated an intent to replace icon 214 in the bottom bar,with the icon 212. As a result of this interaction, icon 212 will appearin the third position of the bottom bar, and icon 214 will be moved backto an overall applications area or box on the desktop. Icon 214 may alsosimply be deleted, in implementations where a copy of the icon is alwayskept in an applications box, regardless of whether the icon appearselsewhere in the system.

FIG. 2B generally shows the manner in which visual elements on a desktopmay be moved and resized, and how such actions affect other elements onthe desktop. The particular displays shown here contain similarelements, but are not meant to indicate a progression of displays thatresult from particular actions by a user.

A contextual actions menu 226 is shown in the figure to provide examplesof various actions that may be performed by a user with respect tographical elements (e.g., program modules and icons) that are part of adesktop. The contextual actions may be inferred by a user, or in certainsituations, such as when a user long presses an element, a menu may popup that is similar to that shown by the contextual actions menu 226 inthe figure. For example, the first menu 226 selection in this example isa selection to rename a box. If a user picks that selection, the activecursor may be moved over the title of the box and the user may beallowed to type in a new name for the box. Another menu 226 selection isa “sort by” selection, which permits the user to sort the icons within abox according to their titles, the type of object to which the icon isrelated, or other such parameters. Yet another selection in the menu 226is a delete selection whose selection by the user erases the entire box.Upon erasing the box, the icons in the box may be eliminated, or may bemoved onto the desktop itself in the space under where they formerly satin the box. Where the icons are eliminated, copies of the icons may bemaintained in a separate applications box that includes all applicationsavailable on the device 220.

Various examples in the figure show results from selecting the secondmenu 226 selection, which is “move & resize.” When a user picks thisselection, they may be given the option of moving a box and/or resizingthe box. Resizing of the box 222 is shown in this example as occurringby use of a handle 224 that appears in a corner of the box after theuser picks the menu selection. The user may press and drag on thehandle, such as through a touch screen input, to change the size oraspect ratio of the box. For example, in the lower right-hand corner ofthe figure, a user is shown as changing a 1×3 box into a 2×2 box thathas one opening for an icon, and then into a 3×1 box. The user couldalso then drag the handle to the left to change the 3×1 box into a 2×1box.

As a second option, the user may choose to move the box 222 rather thanresize it. Such movement may occur, in this example, by the user holdingdown on the title bar of the box 222 and dragging it to anotherlocation. Because each of the various objects in this example are on thesame layer on the desktop it self, other objects that would normally lieunder the box 222 after it has been moved, may be moved and pushed outof the way, and may end up in new locations on the desktop around thebox 222. The movement of the other items may occur in a standardfashion, such as by identifying the closest adjacent open grid locationsinto which the items may be moved. Other adjacent items may also bepushed further away to make room for the items being displaced by themoved box. Also, where the box and the displaced items are of the samesize, the locations for the box and the displaced items may simply beswapped.

FIGS. 2C and 2D show, schematically, techniques for panning across adesktop space. FIG. 2C shows panning in portrait mode, while FIG. 2Dshows panning in landscape mode. In general, a single wallpaper image isprovided for a device 240, whether the device is in landscape mode orportrait mode. The wallpaper, or a portion of the wallpaper, may then bedivided into multiple screen spaces, which may be exclusive of eachother or, in these examples, may be overlapping screen spaces. Inparticular, these displays provided by the device may appear to the useras if the device is a window onto the desktop, where the window may bemoved back and forth over the desktop and new portions of the backgroundand new icons on the desktop may appear as such panning occurs.

Referring more specifically to FIG. 2C, an example wallpaper having afirst dimension 242, and a second dimension 243 is shown schematically.In one example, the first dimension 242 may be 640 pixels, and thesecond dimension 243 may be 480 pixels. Below the wallpaper is shownschematically three different screen spaces for a portrait modepresentation of the wallpaper. A first screen space 242 a shows theleftmost portion of the wallpaper, and is half the width of thewallpaper. The second screen space 242 b is the central half of thewallpaper, and a third screen space 242 c shows the right half of thewallpaper. As shown by device schematic 244, a user may press on thedesktop, such as on an unoccupied area of the desktop where thewallpaper shows through any icons on the desktop, to drag the displayright or left.

The scrolling or panning of the wallpaper relative to icons or otherobjects on the desktop may occur in a parallax manner. In particular,the wallpaper may be moved in a manner so that a larger portion of thewallpaper shows in any given display compared to the desktop objects.For example, the ratio of the wallpaper displayed on any screen may be ½while the ratio of the items on the desktop that are displayed may be ⅓.In this manner, the wallpaper will pan a shorter distance in terms ofpixels for any given panning motion by the user, compared to the panningof the other desktop objects.

As a result, the viewer may be provided with a three-dimensional effectbetween the motion of the objects on the desktop and the motion of thewallpaper, so that the objects appear to float in space closer to theuser, and the wallpaper appears to be more distant from the user. Such aparallax effect may be known to a user from an entirely differentcontext such as video games, in the 1980s video game Moon Patrol, wherethe lunar rover in the foreground and landscaping in the foregroundappear to pan at a rate that is different than mountains in thebackground.

In FIG. 2D, the wallpaper has a first dimension 252 (which may again be640 pixels), and has a second dimension 256 that differs from thecorresponding second dimension 254 of the display. In other words, thebottom portion of an image that generates the wallpaper will be cut offfrom view in landscape mode. In certain circumstances, the image couldalso be compressed, or displayed in other manners. In this way, a singlecommon image can be used for both modes of presentation.

And in this example, each of the screen spaces 252 a-252 c take up alarger portion of the entire displayable area. Again, however, as shownby action 260 on device 258, movement from one screen space to the nextmay be achieved by pressing on the background and dragging. Similarmotion can be achieved using pointer inputs such as trackballs ord-pads.

Panning from one screen space to the next can also be achieved whiledragging an interface element such as an icon. For example, if a usertouches an icon and holds down on it, the user may drag it to the edgeof the screen, and the background may automatically pan over to the nextscreen space, either immediately or after a slight delay that isdesigned to prevent accidental panning. Also, as mentioned, the adjacentscreen spaces may be overlapping (as shown in the figures) ornon-overlapping. Overlapping screen spaces may provide the user withbetter visual continuity, while non-overlapping screen spaces may permita user to pan more quickly across a space.

FIG. 3 is a schematic diagram of a system 300 that provides userinteraction in response to touch screen inputs. The system 300 may beimplemented using a mobile device such as device 302. The device 302includes various input and output mechanisms such as a touch screendisplay 304 and a roller ball 306. A number of components within device302 may be configured to provide various selection functionality ondisplay 304, such as the functionality for organizing a desktopdiscussed above. In addition, the device 302 may be provided with anoperating system, such as the ANDROID operating system, that may provideone or more of the structural components described here.

One such component is a display manager 312, which may be responsiblefor rendering content for presentation on display 304. The displaymanager 312 may receive graphic-related content from a number of sourcesand may determine how the content is to be provided to a user. Forexample, a number of different windows or similar graphical elements forvarious applications 310 on the device 302 may need to be displayed, andthe display manager 312 may determine which to display, which to hide,and what to display or hide when there is overlap between variousgraphical objects.

An input manager 314 may be responsible for translating commandsprovided by a user of device 302. For example, such commands may comefrom a keyboard, from touch screen display 304, from trackball 306, orfrom other such sources, including dedicated buttons or soft buttons(e.g., buttons whose functions may change over time, and whose functionsmay be displayed on areas of display 304 that are adjacent to theparticular buttons). The input manager 314 may determine, for example,in what area of the display commands are being received, and thus inwhat application being shown on the display the commands are intendedfor. In addition, it may interpret input motions on the touch screen 304into a common format and may pass those interpreted motions (e.g., shortpress, long press, multiple press, and straight-line drags) to theappropriate application or to portions of the operating system. Theinput manager 314 may also report such inputs to an event manager (notshown) that in turn reports them to the appropriate modules orapplications.

The input manager 314 may cooperate with the display manager 312 invarious manners, such as to change the displays on the touch screen 304when a user moves from providing input via a pointing device like atrackball to the touch screen itself, or vice versa. As discussed above,input via a pointing device may involve showing a pointer on the touchscreen 304 to indicate where the user is located, and the pointer can beremoved when contact or motion is sensed on the touch screen. In likemanner, objects that have been highlighted as selected objects when auser is employing one input mechanism can be unhighlighted when the userswitches to the other input mechanism (where the input mechanisms are apointing device and the touch screen, or vice-versa). Suchun-highlighting can also occur only in certain situations like thosediscussed above.

A desktop manager 311 may be responsible for managing user interactionswith elements shown to a user on a desktop interface. For example, thedesktop manager 311 may store indicators for each element on a singlelayer of the desktop and coordinate the positioning of such items. Also,the desktop manager 311 may receive alerts when a user selects an item,and may notify the appropriate application (and pass appropriateparameters to the application). For example, if a user selects apersistent search icon, the desktop manager 311 may gather a searchstring associated with the icon, may launch a web browser or independentsearch application, and may submit the search string to the browser orsearch application in the appropriate form to get a remote search engineto return search results to the device 300.

A variety of applications 310 may operate, generally on a commonmicroprocessor, on the device 302. The applications 310 may take avariety of forms, such as mapping applications, e-mail and othermessaging applications, search applications, web browser applications,and various applications running within a web browser or runningextensions of a web browser. The applications may also define userinterface behavior like that described above, for when a user shiftsfrom one input mechanism to another, or the actions may be defined by alibrary that can be accessed by the applications.

A wireless interface 308 managers communication with a wireless network,which may be a data network that also carries voice communications. Thewireless interface may operate in a familiar manner, such as accordingto the examples discussed below, and may provide for communication bythe device 302 with messaging service such as text messaging, e-mail,and telephone voice mail messaging. In addition, the wireless interface308 may support downloads and uploads of content and computer code overa wireless network.

Various forms of persistent storage may be provided, such as using fixeddisk drives and/or solid state memory devices. Two examples are shownhere. First, desktop elements storage 316 includes data representingobjects associated with the desktop. For example, the desktop elementsstorage 316 may contain or refer to image files for icon graphics andtext files for icon and box names. Also, a URI may be associated witheach desktop icon so that selection of the icon calls the URI, andfunctionality contained at the URI can be performed by the operatingsystem. The organization of such referencing may occur in manners likethose described in co-pending U.S. Application Ser. No. 60/987,324,which is herein incorporated by reference in its entirety. The desktopelements storage 316 may also include information about the wallpaperfor a user and the general positioning and layout of items on thedesktop, in addition to indicators of which applications are currentlyactive on the system or were active when the system was last shut down.

Separately, a user profile storage 318, which may be stored on the samemedia as desktop elements storage 316. The user profile 318 includesvarious parameters about a user of the device 302. In the examplerelevant here, the user profile may include data defining a theme fordisplay on the device 302 and other setting peculiar to the user.

FIGS. 4A-4C are flow charts of example processes for interacting withgraphical elements on a computer desktop. In general, FIG. 4A shows aprocess for rendering a desktop when the presentation mode of the adevice changes, e.g., from portrait to landscape or vice-versa. FIG. 4Bgenerally shows a process for responding to user movement of graphicalelements on a desktop, and FIG. 4C generally shows a process forresponding to a user selection of a desktop icon.

Referring now more specifically to FIG. 4A, the example process beginsat box 402, where a mobile device displays various desktop graphicalelements, such as icons, the output of program modules running on thedesktop, and boxes or folders (see FIG. 1A). In box 404, a change inorientation of the device is sensed. Such a change in orientation mayoccur by a number of known mechanisms, such as by sensing an opening ofa slider phone, and determining that such opening indicates an intent totype on a keyboard in a landscape mode. Also, accelerometers or othermotion sensors may be used to determine in orientation of the device. Atbox 406, individual elements are re-rendered for the device, and at box408, the elements are re-spaced on the device. In particular, title barsof icon may be made wider in landscape mode so that additional text mayfit on each title bar, or on each line of each title bar. Also, incertain implementations, the aspect ratio of each icon may be changed ifappropriate, such as by narrowing an icon in the dimension of thedisplay that is currently shortest. In other words, an icon may be madethinner when shown in portrait mode, and shorter or squatter when shownin landscape mode. Re-spacing seeing of the icons and repositioning ofthe icons may occur according to mechanisms such as those shown in FIG.2E, so that the general arrangement of the icons and boxes holding iconson the desktop is maintained between the landscape mode and the portraitmode. In this manner, a user may readily recognize the relativepositioning of their icons very quickly after they switch modes on theirdevice.

At box 410, the bottom bar of the display is moved. For example, thebottom bar may be moved from the bottom of the display in portrait modeto the side of the display in landscape mode. The bottom bar may ingeneral be a docking bar for icons, where the icons are displayed acrossmultiple different screen spaces on the desktop. Generally, a user willplace frequently accessed applications or other objects in the bar sothat they are more readily available to the user, without the userhaving to pick through multiple icons in a box, or pan across a desktoplooking for the icons.

Referring now more specifically to FIG. 4B, there is shown a process forreacting to user movements of elements on a computer desktop. Theprocess begins at box 420, where a user input is received from a user.The user input may include, for example, a tap on an icon, a long presson an icon, and/or a dragging of an icon or flicking of an icon, amongother things. At box 422, the process determines whether the input is amovement, such as a dragging or flicking movement. If the input is not amovement, then an application for the icon may be launched or otheractions defined at a URI associated with the icon may be performed, suchas if the icon input is a clicking or double-clicking input, or acontextual menu for the icon may be raised, such as if the icon input isa long press on the icon.

If the input is a movement, then the process may determine whether thedesktop was a target of the movement, or in other words whether themovement ended on the desktop itself rather than on top of an icon or onsome other element that may be next to or on top of the desktop. If thedesktop was targeted by the movement (box 426), the process may thendetermine whether the an icon that has been moved by the movement wasinitially in a box on the desktop (box 428). If the icon was not in abox, a process may simply move the icon to the desktop (box 430). Forexample, the user may have simply picked up an icon on the desktop andmoved it to a different area on the desktop. Alternatively, the user mayhave picked up on icon from a bottom bar or similar mechanism and movedit onto the desktop. In such a situation, the icon may be removed fromthe bottom bar or from the prior location on the desktop and placed inthe location where the user dropped it.

If the icon was in a box previously, then the process may reorder theicons that are remaining in the box if there is still more than one iconin the box, or may dissolve the box in appropriate circumstances, suchas when a single icon or no icons remain in the box. The size of the boxmay also be adjusted so as to leave a minimum amount of empty spaceinside the box.

If the desktop was not the target of the user's move, the process maydetermine whether an icon on the desktop was a target of the move (box434). Such a motion indicates an intent by the user to create a new boxthat contains the moved icon and the icon that was the target of themovement. Thus, at box 436, the process may form a box for the icons oradd the moved icon to an existing box, if the target icon was already ina box. In certain situations, nesting of boxes may also be provided for,so that dragging an icon on top of another icon in a box will create afurther box within the first box, where the further box includes thedragged icon and the target icon.

If an icon is not a target, then the target in this example would needto be either an area off the edge of the current screen space, or abottom bar or similar structure. Thus, at box 438, the processdetermines whether the motion is off the page and if it is, the processpans the desktop at 440, and then loops back to further determine theultimate target of the user motion. If the motion was not off the page,then the process may add the icon to the bottom bar in the location atwhich the user dropped the item and may move any icon that might havealready been in that location on the bottom bar to an applications boxor may simply delete the icon, as shown at box 442.

Referring now more specifically to FIG. 4C, there is shown a process forexecuting an application or other item represented by an icon (e.g., inthe form of a short-cut) on a computer desktop. As discussed above, eachicon may simply be associated with a URI, and the target of the URI maybe a list of code or fields describing what an author of the code wouldlike performed by the device. One example of such code may be an ANDROIDclass known as an intent. The various actions associated with an intentmay take a wide variety of forms so that icons on a desktop may performa wide range of tasks.

At the initiation of this example process, the icon does not yet exist,so at box 450, the process receives a user request to create a desktopicon. Such a request may be received via a user providing a long pressinput to a background area of a desktop. The process may respond byidentifying a number of standard contexts for such an icon, and maygenerate a pop-up menu over the background to display the contexts tothe user. Some example contexts include icons that: launch a musical orvideo playlist (or perhaps more simply a single musical file), submit apredefined search engine query (e.g., news about Joe Blow in the lastweek), bring up a list of web bookmarks of a particular type or a webpage or group of pages in multiple different tabs in a browser, opens aparticular e-mail folder or a list of e-mail messages that match apredefined search string, raises a mini-application for showing acontact or contacts from a user's contacts list, or other similarfunctions.

The user may then select one of the provided contexts to indicate thetype of function they want performed when the icon is later selected(box 454). They may also then be prompted to enter certaincontext-specific parameters for the icon. For example, if the userchooses a persistent search, they may be asked to enter the searchstring. If the selection is to play music, the user may be asked toselect a particular song or playlist. If the selection is to launch aparticular application, they may be asked for scripting code to beprovided to the application when it launches.

After the user has finished registering their intent, the icon may becreated and assigned a URI (box 456). The URI may, for example, point toa particular playlist that is registered with the operating system. Theparameters provided by the user at box 454 may also be stored at the URIand accessed by the operating system when a user sects an associatedicon. In certain instances, all objects in a system may be assigned aURI, so as to make pointing by icons to URIs a relatively simple task.An image for the icon may then be selected, either automatically (e.g.,according to the type of URI to which the icon points) or by manualselection of the user (box 458).

At some later time, a user may select the icon (box 460). Such selectionmay cause the URI to be provided to the operating system, and theapplication, file, code, or fields, at the URI to be executed (e.g., byeffectively pointing the device to the URI, box 462). The user may thenexperience the functionality associated with the URI.

FIG. 4D is a flow chart of a process for reacting to input from twodifferent input mechanisms on a computing device. In general, theprocess involves changing the display of certain visual elements on atouch screen display when a user moves from providing input from apointing device to providing input on the touch screen itself.

The process begins at box 470 where user input is received at a pointerdevice, such as a joystick, D pad, or trackball. Such motion can occurin various familiar manners. At box 472, the particular motion in thisexample has occurred across a grid of icons on a computer desktop, so asthe user moves the input, different icons are individually highlightedin turn, in the direction of the input motion (e.g., left, right, up, ordown). The selections will jump or snap from one icon to the nextadjacent icon. Also, a user selection may occur via the pointing device(e.g., a click) and multiple icons can be highlighted in that situation.

At box 474, user contact with the touch screen is sensed. Such an eventcan be reported to an application that is managing the current display,and the application may determine that the contact, if in the right areaof the display (e.g., over the icon area), is intended to select iconsin a different manner. As a result, at box 476, the process removes theprior highlighting of the icon or icons, that was in place before thecontact. If the contact is over an icon, it may be highlighted (or anapplication corresponding to the icon can be launched immediatelywithout further action by the user), or if the contact is not over anicon, subsequent dragging motion by the user on the touch screen cancause a selection box to be drawn, where drawing a selection box may nothave been possible using the pointing device. In the latter situation,selecting a group of icons may instead need to occur by highlighting afirst icon, holding down a selection button, and then moving down to asecond icon—where the rectangle defined by the two icons at its cornersmay then have its icons all highlighted (box 480).

Finally, at box 482, a user selection is received and an appropriateoperation is performed on the now-highlighted icons or other objects.For example, if the icons represent applications, each of theapplication can be launched, whereas if the icons represent documents,each of the documents may be raised in its associated application, whichmay be launched if it is not already running.

Referring now to FIG. 5, the exterior appearance of an exemplary device500 that implements the user interface features described here isillustrated. Briefly, and among other things, the device 500 includes aprocessor configured to display notifications regarding events on thedevice 500, and to permit a user to conveniently “pull down” detailabout the events relating to the notifications into an extended view ofthe events.

In more detail, the hardware environment of the device 500 includes adisplay 501 for displaying text, images, and video to a user; a keyboard502 for entering text data and user commands into the device 500; apointing device 504 for pointing, selecting, and adjusting objectsdisplayed on the display 501; an antenna 505; a network connection 506;a camera 507; a microphone 509; and a speaker 510. Although the device500 shows an external antenna 505, the device 500 can include aninternal antenna, which is not visible to the user.

The display 501 can display video, graphics, images, and text that makeup the user interface for the software applications used by the device500, and the operating system programs used to operate the device 500.Among the possible elements that may be displayed on the display 501 area new mail indicator 511 that alerts a user to the presence of a newmessage; an active call indicator 512 that indicates that a telephonecall is being received, placed, or is occurring; a data standardindicator 514 that indicates the data standard currently being used bythe device 500 to transmit and receive data; a signal strength indicator515 that indicates a measurement of the strength of a signal received byvia the antenna 505, such as by using signal strength bars; a batterylife indicator 516 that indicates a measurement of the remaining batterylife; or a clock 517 that outputs the current time.

The display 501 may also show application icons representing variousapplications available to the user, such as a web browser applicationicon 519, a phone application icon 520, a search application icon 521, acontacts application icon 522, a mapping application icon 524, an emailapplication icon 525, or other application icons. In one exampleimplementation, the display 501 is a quarter video graphics array (QVGA)thin film transistor (TFT) liquid crystal display (LCD), capable of16-bit or better color.

A user uses the keyboard (or “keypad”) 502 to enter commands and data tooperate and control the operating system and applications that providefor responding to notification of alerts and responding to messages andthe like (and also to a touch screen). The keyboard 502 includesstandard keyboard buttons or keys associated with alphanumericcharacters, such as keys 526 and 527 that are associated with thealphanumeric characters “Q” and “W” when selected alone, or areassociated with the characters “*” and “1” when pressed in combinationwith key 529. A single key may also be associated with specialcharacters or functions, including unlabeled functions, based upon thestate of the operating system or applications invoked by the operatingsystem. For example, when an application calls for the input of anumeric character, a selection of the key 527 alone may cause a “1” tobe input.

In addition to keys traditionally associated with an alphanumerickeypad, the keyboard 502 also includes other special function keys, suchas an establish call key 530 that causes a received call to be answeredor a new call to be originated; a terminate call key 531 that causes thetermination of an active call; a drop down menu key 532 that causes amenu to appear within the display 501; a backward navigation key 534that causes a previously accessed network address to be accessed again;a favorites key 535 that causes an active web page to be placed in abookmarks folder of favorite sites, or causes a bookmarks folder toappear; a home page key 536 that causes an application invoked on thedevice 500 to navigate to a predetermined network address; or other keysthat provide for multiple-way navigation, application selection, andpower and volume control.

The user uses the pointing device 504 to select and adjust graphics andtext objects displayed on the display 501 as part of the interactionwith and control of the device 500 and the applications invoked on thedevice 500. The pointing device 504 is any appropriate type of pointingdevice, and may be a joystick, a trackball, a touch-pad, a camera, avoice input device, a touch screen device implemented in combinationwith the display 501, or any other input device.

The antenna 505, which can be an external antenna or an internalantenna, is a directional or omni-directional antenna used for thetransmission and reception of radiofrequency (RF) signals that implementpoint-to-point radio communication, wireless local area network (LAN)communication, or location determination. The antenna 505 may facilitatepoint-to-point radio communication using the Specialized Mobile Radio(SMR), cellular, or Personal Communication Service (PCS) frequencybands, and may implement the transmission of data using any number ordata standards. For example, the antenna 505 may allow data to betransmitted between the device 500 and a base station using technologiessuch as Wireless Broadband (WiBro), Worldwide Interoperability forMicrowave ACCess (WiMAX), SGPP Long Term Evolution (LTE), Ultra MobileBroadband (UMB), High Performance Radio Metropolitan Network (HIPERMAN),iBurst or High Capacity Spatial Division Multiple Access (HC-SDMA), HighSpeed OFDM Packet Access (HSOPA), High-Speed Packet Access (HSPA), HSPAEvolution, HSPA+, High Speed Upload Packet Access (HSUPA), High SpeedDownlink Packet Access (HSDPA), Generic Access Network (GAN), TimeDivision-Synchronous Code Division Multiple Access (TD-SCDMA),Evolution-Data Optimized (or Evolution-Data Only)(EVDO), TimeDivision-Code Division Multiple Access (TD-CDMA), Freedom. Of MobileMultimedia Access (FOMA), Universal Mobile Telecommunications System(UMTS), Wideband Code Division Multiple Access (W-CDMA), Enhanced Datarates for GSM Evolution (EDGE), Enhanced GPRS (EGPRS), Code DivisionMultiple Access-2000 (CDMA2000), Wideband Integrated Dispatch EnhancedNetwork (WiDEN), High-Speed Circuit-Switched Data (HSCSD), GeneralPacket Radio Service (GPRS), Personal Handy-Phone System (PHS), CircuitSwitched Data (CSD), Personal Digital Cellular (PDC), CDMAone, DigitalAdvanced Mobile Phone System (D-AMPS), Integrated Digital EnhancedNetwork (IDEN), Global System for Mobile communications (GSM), DataTAC,Mobitex, Cellular Digital Packet Data (CDPD), Hicap, Advanced MobilePhone System (AMPS), Nordic Mobile Phone (NMP), Autoradiopuhelin (ARP),Autotel or Public Automated Land Mobile (PALM), Mobiltelefonisystem D(MTD), Offentlig Landmobil Telefoni (OLT), Advanced Mobile TelephoneSystem (AMTS), Improved Mobile Telephone Service (IMTS), MobileTelephone System (MTS), Push-To-Talk (PTT), or other technologies.Communication via W-CDMA, HSUPA, GSM, GPRS, and EDGE networks may occur,for example, using a QUALCOMM MSM7200A chipset with an QUALCOMM RTR6285transceiver and PM7540 power management circuit.

The wireless or wired computer network connection 506 may be a modemconnection, a local-area network (LAN) connection including theEthernet, or a broadband wide-area network (WAN) connection such as adigital subscriber line (DSL), cable high-speed internet connection,dial-up connection, T-1 line, T-3 line, fiber optic connection, orsatellite connection. The network connection 506 may connect to a LANnetwork, a corporate or government WAN network, the Internet, atelephone network, or other network. The network connection 506 uses awired or wireless connector. Example wireless connectors include, forexample, an INFRARED DATA ASSOCIATION (IrDA) wireless connector, a Wi-Fiwireless connector, an optical wireless connector, an INSTITUTE OFELECTRICAL AND ELECTRONICS ENGINEERS (IEEE) Standard 802.11 wirelessconnector, a BLUETOOTH wireless connector (such as a BLUETOOTH version1.2 or 5.0 connector), a near field communications (NFC) connector, anorthogonal frequency division multiplexing (OFDM) ultra wide band (UWB)wireless connector, a time-modulated ultra wide band (TM-UWB) wirelessconnector, or other wireless connector. Example wired connectorsinclude, for example, a IEEE-1394 FIREWIRE connector, a Universal SerialBus (USB) connector (including a mini-B USB interface connector), aserial port connector, a parallel port connector, or other wiredconnector. In another implementation, the functions of the networkconnection 506 and the antenna 505 are integrated into a singlecomponent.

The camera 507 allows the device 500 to capture digital images, and maybe a scanner, a digital still camera, a digital video camera, otherdigital input device. In one example implementation, the camera 507 is a5 mega-pixel (MP) camera that utilizes a complementary metal-oxidesemiconductor (CMOS).

The microphone 509 allows the device 500 to capture sound, and may be anomni-directional microphone, a unidirectional microphone, abi-directional microphone, a shotgun microphone, or other type ofapparatus that converts sound to an electrical signal. The microphone509 may be used to capture sound generated by a user, for example whenthe user is speaking to another user during a telephone call via thedevice 500. Conversely, the speaker 510 allows the device to convert anelectrical signal into sound, such as a voice from another usergenerated by a telephone application program, or a ring tone generatedfrom a ring tone application program. Furthermore, although the device500 is illustrated in FIG. 5 as a handheld device, in furtherimplementations the device 500 may be a laptop, a workstation, amidrange computer, a mainframe, an embedded system, telephone, desktopPC, a tablet computer, a PDA, or other type of computing device.

FIG. 6 is a block diagram illustrating an internal architecture 600 ofthe device 500. The architecture includes a central processing unit(CPU) 601 where the computer instructions that comprise an operatingsystem or an application are processed; a display interface 602 thatprovides a communication interface and processing functions forrendering video, graphics, images, and texts on the display 501,provides a set of built-in controls (such as buttons, text and lists),and supports diverse screen sizes; a keyboard interface 604 thatprovides a communication interface to the keyboard 502; a pointingdevice interface 605 that provides a communication interface to thepointing device 504; an antenna interface 606 that provides acommunication interface to the antenna 505; a network connectioninterface 607 that provides a communication interface to a network overthe computer network connection 506; a camera interface 608 thatprovides a communication interface and processing functions forcapturing digital images from the camera 507; a sound interface 609 thatprovides a communication interface for converting sound into electricalsignals using the microphone 509 and for converting electrical signalsinto sound using the speaker 510; a random access memory (RAM) 610 wherecomputer instructions and data are stored in a volatile memory devicefor processing by the CPU 601; a read-only memory (ROM) 611 whereinvariant low-level systems code or data for basic system functions suchas basic input and output (I/O), startup, or reception of keystrokesfrom the keyboard 502 are stored in a non-volatile memory device; astorage medium 612 or other suitable type of memory (e.g. such as RAM,ROM, programmable read-only memory (PROM), erasable programmableread-only memory (EPROM), electrically erasable programmable read-onlymemory (EEPROM), magnetic disks, optical disks, floppy disks, harddisks, removable cartridges, flash drives), where the files thatcomprise an operating system 614, application programs 615 (including,for example, a web browser application, a widget or gadget engine, andor other applications, as necessary) and data files 616 are stored; anavigation module 617 that provides a real-world or relative position orgeographic location of the device 500; a power source 619 that providesan appropriate alternating current (AC) or direct current (DC) to powercomponents; and a telephony subsystem 620 that allows the device 500 totransmit and receive sound over a telephone network. The constituentdevices and the CPU 601 communicate with each other over a bus 621.

The CPU 601 can be one of a number of computer processors. In onearrangement, the computer CPU 601 is more than one processing unit. TheRAM 610 interfaces with the computer bus 621 so as to provide quick RAMstorage to the CPU 601 during the execution of software programs such asthe operating system application programs, and device drivers. Morespecifically, the CPU 601 loads computer-executable process steps fromthe storage medium 612 or other media into a field of the RAM 610 inorder to execute software programs. Data is stored in the RAM 610, wherethe data is accessed by the computer CPU 601 during execution. In oneexample configuration, the device 500 includes at least 128 MB of RAM,and 256 MB of flash memory.

The storage medium 612 itself may include a number of physical driveunits, such as a redundant array of independent disks (RAID), a floppydisk drive, a flash memory, a USB flash drive, an external hard diskdrive, thumb drive, pen drive, key drive, a High-Density DigitalVersatile Disc (HD-DVD) optical disc drive, an internal hard disk drive,a Blu-Ray optical disc drive, or a Holographic Digital Data Storage(HDDS) optical disc drive, an external mini-dual in-line memory module(DIMM) synchronous dynamic random access memory (SDRAM), or an externalmicro-DIMM SDRAM. Such computer readable storage media allow the device500 to access computer-executable process steps, application programsand the like, stored on removable and non-removable memory media, tooff-load data from the device 500, or to upload data onto the device500.

A computer program product is tangibly embodied in storage medium 612, amachine-readable storage medium. The computer program product includesinstructions that, when read by a machine, operate to cause a dataprocessing apparatus to store image data in the mobile device. In someembodiments, the computer program product includes instructions thatgenerate notifications about alerts such as newly arriving messages onthe device.

The operating system 614 may be a LINUX-based operating system such asthe GOOGLE mobile device platform; APPLE MAC OS X; MICROSOFT WINDOWSNT/WINDOWS 2000/WINDOWS XP/WINDOWS MOBILE; a variety of UNIX-flavoredoperating systems; or a proprietary operating system for computers orembedded systems. The application development platform or framework forthe operating system 614 may be: BINARY RUNTIME ENVIRONMENT FOR WIRELESS(BREW); JAVA Platform, Micro Edition (JAVA ME) or JAVA 2 Platform, MicroEdition (J2ME) using the SUN MICROSYSTEMS JAVASCRIPT programminglanguage; PYTHON FLASH LITE, or MICROSOFT .NET Compact, or anotherappropriate environment.

The device stores computer-executable code for the operating system 614,and the application programs 615 such as an email, instant messaging, avideo service application, a mapping application word processing,spreadsheet, presentation, gaming, mapping, web browsing, JAVASCRIPTengine, or other applications. For example, one implementation may allowa user to access the GOOGLE GMAIL email application, the GOOGLE TALKinstant messaging application, a YOUTUBE video service application, aGOOGLE MAPS or GOOGLE EARTH mapping application, or a GOOGLE PICASAimaging editing and presentation application. The application programs615 may also include a widget or gadget engine, such as a TAFRI widgetengine, a MICROSOFT gadget engine such as the WINDOWS SIDEBAR gadgetengine or the KAPSULES gadget engine, a YAHOO! widget engine such as theKONFABULTOR widget engine, the APPLE DASHBOARD widget engine, the GOOGLEgadget engine, the KLIPFOLIO widget engine, an OPERA widget engine, theWIDSETS widget engine, a proprietary widget or gadget engine, or otherwidget or gadget engine the provides host system software for aphysically-inspired applet on a desktop.

Although it is possible to provide for notifications and interactionswith messages and other events using the above-described implementation,it is also possible to implement the functions according to the presentdisclosure as a dynamic link library (DLL), or as a plug-in to otherapplication programs such as an Internet web-browser such as the FOXFIREweb browser, the APPLE SAFARI web browser or the MICROSOFT INTERNETEXPLORER web browser.

The navigation module 617 may determine an absolute or relative positionof the device, such as by using the Global Positioning System (GPS)signals, the GLObal NAvigation Satellite System (GLONASS), the Galileopositioning system, the Beidou Satellite Navigation and PositioningSystem, an inertial navigation system, a dead reckoning system, or byaccessing address, internet protocol (IP) address, or locationinformation in a database. The navigation module 617 may also be used tomeasure angular displacement, orientation, or velocity of the device500, such as by using one or more accelerometers.

FIG. 7 is a block diagram illustrating exemplary components of theoperating system 614 used by the device 500, in the case where theoperating system 614 is the GOOGLE mobile device platform. The operatingsystem 614 invokes multiple processes, while ensuring that theassociated phone application is responsive, and that waywardapplications do not cause a fault (or “crash”) of the operating system.Using task switching, the operating system 614 allows for the switchingof applications while on a telephone call, without losing the state ofeach associated application. The operating system 614 may use anapplication framework to encourage reuse of components, and provide ascalable user experience by combining pointing device and keyboardinputs and by allowing for pivoting. Thus, the operating system canprovide a rich graphics system and media experience, while using anadvanced, standards-based web browser.

The operating system 614 can generally be organized into six components:a kernel 700, libraries 701, an operating system runtime 702,application libraries 704, system services 705, and applications 706.The kernel 700 includes a display driver 707 that allows software suchas the operating system 614 and the application programs 715 to interactwith the display 501 via the display interface 602, a camera driver 709that allows the software to interact with the camera 507; a BLUETOOTHdriver 710; a M-Systems driver 711; a binder (IPC) driver 712, a USBdriver 714 a keypad driver 715 that allows the software to interact withthe keyboard 502 via the keyboard interface 604; a WiFi driver 716;audio drivers 717 that allow the software to interact with themicrophone 509 and the speaker 510 via the sound interface 609; and apower management component 719 that allows the software to interact withand manage the power source 719.

The BLUETOOTH driver, which in one implementation is based on the BlueZBLUETOOTH stack for LINUX-based operating systems, provides profilesupport for headsets and hands-free devices, dial-up networking,personal area networking (PAN), or audio streaming (such as by AdvanceAudio Distribution Profile (A2DP) or Audio/Video Remote Control Profile(AVRCP). The BLUETOOTH driver provides JAVA bindings for scanning,pairing and unpairing, and service queries.

The libraries 701 include a media framework 720 that supports standardvideo, audio and still-frame formats (such as Moving Picture ExpertsGroup (MPEG)-4, H.264, MPEG-1 Audio Layer-3 (MP3), Advanced Audio Coding(AAC), Adaptive Multi-Rate (AMR), Joint Photographic Experts Group(JPEG), and others) using an efficient JAVA Application ProgrammingInterface (API) layer; a surface manager 721; a simple graphics library(SGL) 722 for two-dimensional application drawing; an Open GraphicsLibrary for Embedded Systems (OpenGL ES) 724 for gaming andthree-dimensional rendering; a C standard library (LIBC) 725; aLIBWEBCORE library 726; a FreeType library 727; an SSL 729; and anSQLite library 730.

The operating system runtime 702 includes core JAVA libraries 731, and aDalvik virtual machine 732. The Dalvik virtual machine 732 is a custom,virtual machine that runs a customized file format (.DEX).

The operating system 614 can also include Mobile Information DeviceProfile (MIDP) components such as the MIDP JAVA Specification Requests(JSRs) components, MIDP runtime, and MIDP applications as shown in FIG.7. The MIDP components can support MIDP applications running on thedevice 500.

With regard to graphics rendering, a system-wide composer managessurfaces and a frame buffer and handles window transitions, using theOpenGL ES 724 and two-dimensional hardware accelerators for itscompositions.

The Dalvik virtual machine 732 may be used with an embedded environment,since it uses runtime memory very efficiently, implements aCPU-optimized bytecode interpreter, and supports multiple virtualmachine processes per device. The custom file format (DEX) is designedfor runtime efficiency, using a shared constant pool to reduce memory,read-only structures to improve cross-process sharing, concise, andfixed-width instructions to reduce parse time, thereby allowinginstalled applications to be translated into the custom file formal atbuild-time. The associated bytecodes are designed for quickinterpretation, since register-based instead of stack-based instructionsreduce memory and dispatch overhead, since using fixed widthinstructions simplifies parsing, and since the 16-bit code unitsminimize reads.

The application libraries 704 include a view system 734, a resourcemanager 735, and content providers 737. The system services 705 includesa status bar 739; an application launcher 740; a package manager 741that maintains information for all installed applications; a telephonymanager 742 that provides an application level JAVA interface to thetelephony subsystem 620; a notification manager 744 that allows allapplications access to the status bar and on-screen notifications; awindow manager 745 that allows multiple applications with multiplewindows to share the display 501; and an activity manager 746 that runseach application in a separate process, manages an application lifecycle, and maintains a cross-application history.

The applications 706 include a home application 747, a dialerapplication 749, a contacts application 750, and a browser application751. Each of the applications may generate graphical elements thateither do or do not have long press interactions. As described above,those that do not have long press interactions may provide no immediatevisual feedback when they are first pressed, while those that do havesuch interactions may be highlighted between the time they are firstpressed and the expiration of the long press period. Also, thehighlighting may not occur exactly upon a press, so that mere tapping ofan item does not cause it to be highlighted; instead, the highlightingmay occur upon the expiration of a short press period that is slightlymore than the time period for a tap, but appreciably shorter than a longpress period.

The telephony manager 742 provides event notifications (such as phonestate, network state, Subscriber Identity Module (SIM) status, orvoicemail status), allows access to state information (such as networkinformation, SIM information, or voicemail presence), initiates calls,and queries and controls the call state. The browser application 751renders web pages in a full, desktop-like manager, including navigationfunctions. Furthermore, the browser application 751 allows singlecolumn, small screen rendering, and provides for the embedding of HTMLviews into other applications.

FIG. 8 is a block diagram illustrating exemplary processes implementedby the operating system kernel 800. Generally, applications and systemservices run in separate processes, where the activity manager 746 runseach application in a separate process and manages the application lifecycle. The applications run in their own processes, although manyactivities or services can also run in the same process. Processes arestarted and stopped as needed to run an application's components, andprocesses may be terminated to reclaim resources. Each application isassigned its own process, whose name is the application's package name,and individual parts of an application can be assigned another processname.

Some processes can be persistent. For example, processes associated withcore system components such as the surface manager 816, the windowmanager 814, or the activity manager 810 can be continuously executedwhile the device 500 is powered. Additionally, some application-specificprocess can also be persistent. For example, processes associated withthe dialer application 821, may also be persistent.

The processes implemented by the operating system kernel 800 maygenerally be categorized as system services processes 801, dialerprocesses 802, browser processes 804, and maps processes 805. The systemservices processes 801 include status bar processes 806 associated withthe status bar 739; application launcher processes 807 associated withthe application launcher 740; package manager processes 809 associatedwith the package manager 741; activity manager processes 810 associatedwith the activity manager 746; resource manager processes 811 associatedwith a resource manager 811 that provides access to graphics, localizedstrings, and XML layout descriptions; notification manger processes 812associated with the notification manager 744; window manager processes814 associated with the window manager 745; core JAVA librariesprocesses 815 associated with the core JAVA libraries 731; surfacemanager processes 816 associated with the surface manager 721; Dalvikvirtual machine processes 817 associated with the Dalvik virtual machine732, and LIBC processes 819 associated with the LIBC library 725.

The dialer processes 802 include dialer application processes 821associated with the dialer application 749; telephony manager processes822 associated with the telephony manager 742; core JAVA librariesprocesses 824 associated with the core JAVA libraries 731; Dalvikvirtual machine processes 825 associated with the Dalvik Virtual machine732; and LIBC processes 826 associated with the LIBC library 725. Thebrowser processes 804 include browser application processes 827associated with the browser application 751; core JAVA librariesprocesses 829 associated with the core JAVA libraries 731; Dalvikvirtual machine processes 830 associated with the Dalvik virtual machine732; LIBWEBCORE processes 831 associated with the LIBWEBCORE library726; and LIBC processes 832 associated with the LIBC library 725.

The maps processes 805 include maps application processes 834, core JAVAlibraries processes 835, Dalvik virtual machine processes 836, and LIBCprocesses 837. Notably, some processes, such as the Dalvik virtualmachine processes, may exist within one or more of the systems servicesprocesses 801, the dialer processes 802, the browser processes 804, andthe maps processes 805.

FIG. 9 shows an example of a generic computer device 900 and a genericmobile computer device 950, which may be used with the techniquesdescribed here. Computing device 900 is intended to represent variousforms of digital computers, such as laptops, desktops, workstations,personal digital assistants, servers, blade servers, mainframes, andother appropriate computers. Computing device 950 is intended torepresent various forms of mobile devices, such as personal digitalassistants, cellular telephones, smartphones, and other similarcomputing devices. The components shown here, their connections andrelationships, and their functions, are meant to be exemplary only, andare not meant to limit implementations of the inventions describedand/or claimed in this document.

Computing device 900 includes a processor 902, memory 904, a storagedevice 906, a high-speed interface 908 connecting to memory 904 andhigh-speed expansion ports 910, and a low speed interface 912 connectingto low speed bus 914 and storage device 906. Each of the components 902,904, 906, 908, 910, and 912, are interconnected using various busses,and may be mounted on a common motherboard or in other manners asappropriate. The processor 902 can process instructions for executionwithin the computing device 900, including instructions stored in thememory 904 or on the storage device 906 to display graphical informationfor a GUI on an external input/output device, such as display 916coupled to high speed interface 908. In other implementations, multipleprocessors and/or multiple buses may be used, as appropriate, along withmultiple memories and types of memory. Also, multiple computing devices900 may be connected, with each device providing portions of thenecessary operations (e.g., as a server bank, a group of blade servers,or a multi-processor system).

The memory 904 stores information within the computing device 900. Inone implementation, the memory 904 is a volatile memory unit or units.In another implementation, the memory 904 is a non-volatile memory unitor units. The memory 904 may also be another form of computer-readablemedium, such as a magnetic or optical disk.

The storage device 906 is capable of providing mass storage for thecomputing device 900. In one implementation, the storage device 906 maybe or contain a computer-readable medium, such as a floppy disk device,a hard disk device, an optical disk device, or a tape device, a flashmemory or other similar solid state memory device, or an array ofdevices, including devices in a storage area network or otherconfigurations. A computer program product can be tangibly embodied inan information carrier. The computer program product may also containinstructions that, when executed, perform one or more methods, such asthose described above. The information carrier is a computer- ormachine-readable medium, such as the memory 904, the storage device 906,memory on processor 902, or a propagated signal.

The high speed controller 908 manages bandwidth-intensive operations forthe computing device 900, while the low speed controller 912 manageslower bandwidth-intensive operations. Such allocation of functions isexemplary only. In one implementation, the high-speed controller 908 iscoupled to memory 904, display 916 (e.g., through a graphics processoror accelerator), and to high-speed expansion ports 910, which may acceptvarious expansion cards (not shown). In the implementation, low-speedcontroller 912 is coupled to storage device 906 and low-speed expansionport 914. The low-speed expansion port, which may include variouscommunication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet)may be coupled to one or more input/output devices, such as a keyboard,a pointing device, a scanner, or a networking device such as a switch orrouter, e.g., through a network adapter.

The computing device 900 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as astandard server 920, or multiple times in a group of such servers. Itmay also be implemented as part of a rack server system 924. Inaddition, it may be implemented in a personal computer such as a laptopcomputer 922. Alternatively, components from computing device 900 may becombined with other components in a mobile device (not shown), such asdevice 950. Each of such devices may contain one or more of computingdevice 900, 950, and an entire system may be made up of multiplecomputing devices 900, 950 communicating with each other.

Computing device 950 includes a processor 952, memory 964, aninput/output device such as a display 954, a communication interface966, and a transceiver 968, among other components. The device 950 mayalso be provided with a storage device, such as a microdrive or otherdevice, to provide additional storage. Each of the components 950, 952,964, 954, 966, and 968, are interconnected using various buses, andseveral of the components may be mounted on a common motherboard or inother manners as appropriate.

The processor 952 can execute instructions within the computing device950, including instructions stored in the memory 964. The processor maybe implemented as a chipset of chips that include separate and multipleanalog and digital processors. The processor may provide, for example,for coordination of the other components of the device 950, such ascontrol of user interfaces, applications run by device 950, and wirelesscommunication by device 950.

Processor 952 may communicate with a user through control interface 958and display interface 956 coupled to a display 954. The display 954 maybe, for example, a TFT LCD (Thin-Film-Transistor Liquid Crystal Display)or an OLED (Organic Light Emitting Diode) display, or other appropriatedisplay technology. The display interface 956 may comprise appropriatecircuitry for driving the display 954 to present graphical and otherinformation to a user. The control interface 958 may receive commandsfrom a user and convert them for submission to the processor 952. Inaddition, an external interface 962 may be provide in communication withprocessor 952, so as to enable near area communication of device 950with other devices. External interface 962 may provide, for example, forwired communication in some implementations, or for wirelesscommunication in other implementations, and multiple interfaces may alsobe used.

The memory 964 stores information within the computing device 950. Thememory 964 can be implemented as one or more of a computer-readablemedium or media, a volatile memory unit or units, or a non-volatilememory unit or units. Expansion memory 974 may also be provided andconnected to device 950 through expansion interface 972, which mayinclude, for example, a SIMM (Single In Line Memory Module) cardinterface. Such expansion memory 974 may provide extra storage space fordevice 950, or may also store applications or other information fordevice 950. Specifically, expansion memory 974 may include instructionsto carry out or supplement the processes described above, and mayinclude secure information also. Thus, for example, expansion memory 974may be provide as a security module for device 950, and may beprogrammed with instructions that permit secure use of device 950. Inaddition, secure applications may be provided via the SIMM cards, alongwith additional information, such as placing identifying information onthe SIMM card in a non-hackable manner.

The memory may include, for example, flash memory and/or NVRAM memory,as discussed below. In one implementation, a computer program product istangibly embodied in an information carrier. The computer programproduct contains instructions that, when executed, perform one or moremethods, such as those described above. The information carrier is acomputer- or machine-readable medium, such as the memory 964, expansionmemory 974, memory on processor 952, or a propagated signal that may bereceived, for example, over transceiver 968 or external interface 962.

Device 950 may communicate wirelessly through communication interface966, which may include digital signal processing circuitry wherenecessary. Communication interface 966 may provide for communicationsunder various modes or protocols, such as GSM voice calls, SMS, EMS, orMMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others.Such communication may occur, for example, through radio-frequencytransceiver 968. In addition, short-range communication may occur, suchas using a Bluetooth, WiFi, or other such transceiver (not shown). Inaddition, GPS (Global Positioning System) receiver module 970 mayprovide additional navigation- and location-related wireless data todevice 950, which may be used as appropriate by applications running ondevice 950.

Device 950 may also communicate audibly using audio codec 960, which mayreceive spoken information from a user and convert it to usable digitalinformation. Audio codec 960 may likewise generate audible sound for auser, such as through a speaker, e.g., in a handset of device 950. Suchsound may include sound from voice telephone calls, may include recordedsound (e.g., voice messages, music files, etc.) and may also includesound generated by applications operating on device 950.

The computing device 950 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as acellular telephone 980. It may also be implemented as part of asmartphone 982, personal digital assistant, or other similar mobiledevice.

Various implementations of the systems and techniques described here canbe realized in digital electronic circuitry, integrated circuitry,specially designed ASICs (application specific integrated circuits),computer hardware, firmware, software, and/or combinations thereof.These various implementations can include implementation in one or morecomputer programs that are executable and/or interpretable on aprogrammable system including at least one programmable processor, whichmay be special or general purpose, coupled to receive data andinstructions from, and to transmit data and instructions to, a storagesystem, at least one input device, and at least one output device.

These computer programs (also known as programs, software, softwareapplications or code) include machine instructions for a programmableprocessor, and can be implemented in a high-level procedural and/orobject-oriented programming language, and/or in assembly/machinelanguage. As used herein, the terms “machine-readable medium”“computer-readable medium” refers to any computer program product,apparatus and/or device (e.g., magnetic discs, optical disks, memory,Programmable Logic Devices (PLDs)) used to provide machine instructionsand/or data to a programmable processor, including a machine-readablemedium that receives machine instructions as a machine-readable signal.The term “machine-readable signal” refers to any signal used to providemachine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniquesdescribed here can be implemented on a computer having a display device(e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor)for displaying information to the user and a keyboard and a pointingdevice (e.g., a mouse or a trackball) by which the user can provideinput to the computer. Other kinds of devices can be used to provide forinteraction with a user as well; for example, feedback provided to theuser can be any form of sensory feedback (e.g., visual feedback,auditory feedback, or tactile feedback); and input from the user can bereceived in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in acomputing system that includes a back end component (e.g., as a dataserver), or that includes a middleware component (e.g., an applicationserver), or that includes a front end component (e.g., a client computerhaving a graphical user interface or a Web browser through which a usercan interact with an implementation of the systems and techniquesdescribed here), or any combination of such back end, middleware, orfront end components. The components of the system can be interconnectedby any form or medium of digital data communication (e.g., acommunication network). Examples of communication networks include alocal area network (“LAN”), a wide area network (“WAN”), and theInternet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

A number of embodiments have been described. Nevertheless, it will beunderstood that various modifications may be made without departing fromthe spirit and scope of the invention. For example, much of thisdocument has been described with respect to messaging and mappingapplications, but other forms of graphical applications may also beaddressed, such as interactive program guides, web page navigation andzooming, and other such applications.

In addition, the logic flows depicted in the figures do not require theparticular order shown, or sequential order, to achieve desirableresults. In addition, other steps may be provided, or steps may beeliminated, from the described flows, and other components may be addedto, or removed from, the described systems. Accordingly, otherembodiments are within the scope of the following claims.

1-20. (canceled) 21: A method comprising: outputting, by a computingdevice and for display at a proximity sensitive display, a graphicaluser interface including a plurality of user interface elements;detecting, by the proximity sensitive display, a user input toreposition a first user interface element from the plurality of userinterface elements onto a location of the graphical user interface thatis occupied by a second user interface element from the plurality ofuser interface elements; responsive to detecting the user input,determining, by the computing device, whether the second user interfaceelement of the graphical user interface is a target of the reposition;and responsive to determining that the second user interface element isthe target of the reposition, displaying, by the proximity sensitivedisplay, a folder within the graphical user interface at the location ofthe graphical user interface that was occupied by the second userinterface element, the folder including the first user interface elementand the second user interface element. 22: The method of claim 21,wherein the user input to reposition the first user interface elementcomprises a press user input followed by a drag user input, the methodfurther comprising: responsive to determining that the press user inputoccurs for at least a threshold duration: visually indicating that thefirst user interface element is capable of being moved; and moving,based on the drag user input, the first user interface element withinthe graphical user interface. 23: The method of claim 21, wherein thesecond user interface element is included within a first folder, andwherein the folder is a second folder visually nested inside the firstfolder such that the second folder includes both the first userinterface element and the second user interface element and the firstfolder includes the second folder. 24: The method of claim 21, whereinthe graphical user interface includes a plurality of screen spaces, themethod further comprising: responsive to determining that a desktop ofthe graphical user interface is a target of the repositioning:repositioning the first user interface element onto a location on afirst screen space from the plurality of screen spaces, wherein thefirst screen space is visible on the proximity sensitive display at agiven time, and is occupied by the second user interface element;identifying a location on a second screen space from the plurality ofscreen spaces, wherein the second screen space is not visible on thedisplay at the given time, and is not occupied by any other userinterface element from the plurality of user interface elements; andrepositioning the second user interface element from the location on thefirst screen space onto the identified location on the second screenspace. 25: The method of claim 21, further comprising: providing, by thecomputing device and during the repositioning, tactile feedback as thefirst user interface element is moved from an initial location, acrosseach location in a grid of the graphical user interface, and to thelocation of the graphical user interface that is occupied by the seconduser interface element. 26: The method of claim 21, wherein: thegraphical user interface further includes a bottom bar having at leastone user interface element from the plurality of user interfaceelements; the first user interface element is located within the bottombar prior to the user input to reposition the first user interfaceelement; and the method further comprises, after detecting the userinput, removing the first user interface element from the bottom bar.27: The method of claim 21, wherein: the folder is a second folderwithin the graphical user interface; the first user interface element isinitially located in a first folder within the graphical user interface;and the method further comprises, after detecting the user input,removing the first user interface element from the first folder. 28: Themethod of claim 27, wherein: the first user interface element is theonly user interface element located in the first folder prior todetecting the user input; and the method further comprises, afterdetecting the user input, removing, from the graphical user interface,the first folder. 29: The method of claim 27, wherein, after removingthe first user interface element from the first folder, there are atleast two remaining user interface elements located in the first folder,the method further comprising rearranging the at least two remaininguser interface elements within the first folder. 30: The method of claim27, wherein after removing the first user interface element from thefirst folder, there is one remaining user interface element located inthe first folder, the method further comprising: removing the firstfolder from the graphical user interface; and displaying the oneremaining user interface element in place of the first folder within thegraphical user interface. 31: A computing device comprising: a proximitysensitive display that displays a graphical user interface including aplurality of user interface elements, and detects a user input toreposition a first user interface element from the plurality of userinterface elements onto a location of the graphical user interface thatis occupied by a second user interface element from the plurality ofuser interface elements, wherein the user input is a press user inputfollowed by a drag user input; one or more processors; and anon-transitory computer-readable medium configured to store instructionsthat, when executed by the one or more processors, cause the one or moreprocessors to: responsive to the proximity sensitive display detectingthe press user input for at least a threshold duration: output, fordisplay by the proximity sensitive display, a visual indication that thefirst user interface element is capable of being moved; and output, fordisplay by the proximity sensitive display and based on the drag input,movement of the first user interface element within the graphical userinterface; determine whether the second user interface element of thegraphical user interface is a target of the reposition; and responsiveto determining that the second user interface element is the target ofthe reposition, output, for display by the proximity sensitive display,a folder within the graphical user interface at the location of thegraphical user interface that was occupied by the second user interfaceelement, the folder including the first user interface element and thesecond user interface element. 32: The computing device of claim 31,wherein: the graphical user interface further includes a bottom barhaving at least one user interface element from the plurality of userinterface elements; the first user interface element is located withinthe bottom bar prior to the user input to reposition the first userinterface element; and the instructions, when executed, cause the one ormore processors to, after the proximity sensitive display detects theuser input, output, for display by the proximity sensitive display, anupdated graphical user interface that does not include the first userinterface element in the bottom bar. 33: The computing device of claim31, wherein the folder is a second folder within the graphical userinterface; the first user interface element is initially located in afirst folder within the graphical user interface; and the instructions,when executed, cause the one or more processors to, after the proximitysensitive display detects the user input, output, for display by theproximity sensitive display, an updated graphical user interface doesnot include the first user interface element in the first folder. 34:The computing device of claim 33, wherein: the first user interfaceelement is the only user interface element located in the first folderprior to detecting the user input; and the updated graphical userinterface does not include the first folder. 35: The computing device ofclaim 33, wherein: after removing the first user interface element fromthe first folder, there are at least two remaining user interfaceelements located in the first folder, the updated graphical userinterface includes the at least two remaining user interface elementsrearranged within the first folder. 36: A non-transitorycomputer-readable storage medium encoded with instructions that, whenexecuted by one or more processors of a computing device, cause the oneor more processors to perform operations comprising: providing, fordisplay at a proximity sensitive display, a graphical user interfaceincluding a plurality of user interface elements; receiving anindication of a user input to reposition a first user interface elementfrom the plurality of user interface elements onto a location of thegraphical user interface that is occupied by a second user interfaceelement from the plurality of user interface elements, wherein the userinput is detected by the proximity sensitive display; responsive toreceiving the indication of the user input, determining whether thesecond user interface element of the graphical user interface is atarget of the reposition; and responsive to determining that the seconduser interface element is the target of the reposition, providing, fordisplay by the proximity sensitive display, a folder within thegraphical user interface at the location of the graphical user interfacethat was occupied by the second user interface element, the folderincluding the first user interface element and the second user interfaceelement. 37: The non-transitory computer-readable storage medium ofclaim 36, wherein: the graphical user interface further includes abottom bar having at least one user interface element from the pluralityof user interface elements; the first user interface element is locatedwithin the bottom bar prior to the user input to reposition the firstuser interface element; and the operations further comprise, afterreceiving the indication of the user input, output, for display by theproximity sensitive display, an updated graphical user interface thatdoes not include the first user interface element in the bottom bar. 38:The non-transitory computer-readable storage medium of claim 36, whereinthe folder is a second folder within the graphical user interface; thefirst user interface element is initially located in a first folderwithin the graphical user interface; and the operations furthercomprise, after receiving the indication of the user input, output, fordisplay by the proximity sensitive display, an updated graphical userinterface that does not include the first user interface element in thefirst folder. 39: The non-transitory computer-readable storage medium ofclaim 38, wherein: the first user interface element is the only userinterface element located in the first folder prior to detecting theuser input; and the updated graphical user interface does not includethe first folder. 40: The non-transitory computer-readable storagemedium of claim 38, wherein: after removing the first user interfaceelement from the first folder, there are at least two remaining userinterface elements located in the first folder, the updated graphicaluser interface includes the at least two remaining user interfaceelements rearranged within the first folder.